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3.8 REPORT OF THE SUBPANEL ON METHODS OF VERIFICATION* 


The methods of verification subpanel met for two ^-hour sessins on November 
18 and 19# 1981, to define research needs ir this area for Earth resource ob- 
servational systems • The panel consisted of: 


John C* Lyon 
GSFC 

Maria Kalcic 
NASA/NSTL 

B.R. Seyfarth 
NASA/NSTL 

Johh Sos 
NASA/GSFC 

Gerald Grebowsky 
NASA/GSFC 


John Snyde 

uses 

Ai Chun Fang 
NASA Headquarters 

R.B. MacDonald 
NASA/ JSC 

William P, Clark 
NASA/GSFC 

Len Gaydos 

uses 


3.8.1 State of Knowledge 

We refer to verification as a production oriented question because we are not 
really doing verification for the uses of research activities. We*r«^ trying 
to verify output data products from a production system for production ori- 
ented users. Included in that class of production oriented users is the pro- 
duction system itself, i.e., quality control of output. Verification is not a 
well defined problem. We’re just beginning to focus on understanding what 
we’re talking about. Therefore we recommend a program tc improve the state of 
understanding and of the meaning of verification and the application of veri- 
fication procedures to a variety of sensor systems. 

3.8.2 Recommended Research 

The topics of interest that were identified are very strongly interrelated. 
To show this we have developed a task flow in Figures 1, 2, and 3. Other than 
having done that, it war found that there is no real priority scheme that 
could be assigned, as this is an integrated activity designed to provide more 
understanding into what needs to be done in imag'' rectification. Task one 
(Figure 1) is to develop verification procedures. Task two (Figure 2) in- 
volves an experimental hands-on data demonstration and evaluation of those 
procedures in a controlled test-bed experiment. For task one, the flow is as 
follows: you first need to identify what performance measures you’re really 
concerned about. We’ve suggested a handful of starters such as the number of 
control points, residual correlation mapped across an image, and the distribu- 
tion of errors within the image plane for individual and coincident sets of 
images. Using that information, it Is possible to gc directly to existing 
data and compile real and synthetic data sets that can address that problem. 
In parallel, one can review the current verification procedures that are used. 


•Edited oral presentation. 
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This is not a trivial undertaking in spite of the fact that the current veri- 
fication procedures are very casual and unsystematic. There are some 
activties going on in other fields, in mapping especially, that have orderly 
procedures that are not exactly remote sensing oriented procedures but can be 
training grounds for a subsepr^nc task. From the existing procedures, identi- 
fy and recommend the ones on hand that are usable in an operational system, 
develop additional techniques necessary with some synthetic data sets, and 
perform specific tests on individual package procedures to analyse the 
results. The end result then is a set of verfication p»-ocedures and software, 
or at least algorithms, that can be useable. The oucput of that then goes to 
task two, which is the verification experiment. 

Task two is the end-to-end test experiment. ^e very first step in task two 
is to identify the test bed. There are several possible options and it’s a 
matter of timing and some other subtle details that relate to exactly what 
procedures and requirements you want to develop. The output of the first part 
of task one delineates what verification you really want to do that will help 
to choose a test-bed. The obvious choices on-hand are Landsat MSS data or, in 
the nerr future, the Landsat-D data processing capabilities. The next step is 
to design the experiment to test verification procedures. This procedure in 
development and selection activity is from task one, as it is essentially task 
one that feeds into that knowledge of the sensor system itself, including er- 
ror budgets, as a part of establishing the design. Then it’s a natter of im- 
plementing the experiment. The output of doing software oriented measurements 
on standardized data sets allows one to perform some accuracy measurements. 
You want to pay very close attention to the feedback loop so that when you get 
part way there you start looKing at the users on the other side of the infor- 
mation system that need to see the results to help better understand what they 
really wanted to have asked you for in verification. That finally leads you 
to moving one step beyond to the internal problem and you start deriving the 
error sources observed and finally report and assess the results of this 
experiment. 

Tables I and II outline in summary form the required elements for each task 
described . 
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TASK 1 

DEVELOP VEKIFICATION PROCEDURES 
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ANALYZE DERIVED VERIFICATION REQUIREMENTS VS CURRENT APPROACHES 
IDENTIFY VERIFICATION APPROACHES 



DEVELOP ADUrriONAL TECHNIQUES 


OF POOR QUALiJt 
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IMAGE VEKIFICATION END-TO-END TEST 


ORIGINAL PAGc i3 
OF POOR QUALITY 
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0 USER CHECK OF- ACCURACY REPORT 

0 SENSOR SYSTEM DEVELOPER OPINION OF RECOMMENDED CHANGES 
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FIGURE 1. TASK ONE: ACTIVITY FLOW 
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FIGURE 2. 


IMAGE VERl. ICATION END-TO-END TESTING 
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PRESENTATIONS ON USER NEEDS 


Y 


/ 


n.o 

i*.i INTRODUCTION 

It was the pu-pose of the User Needs presentations to discuss the implication^ 
of satellite image positional accuracy capabilities upon their discipline's 
use of the data. Discipline experts discussed the manner in which data are 
incorporated into their inventory, decision, or analysis models and opera- 
tions. For each discipline area, monitoring and data base integration stra- 
tegic’s we»'e discussed; in particular, implications for registration and recti- 
fication requirements. In addition, the discussants pointed out the potential 
benefit that might accrue from improved registration/rectification accuracy. 
Were appi cable, the discipline experts discussed their desire to implanent 
muititemporal, multistage, and multisensor data sources for addressing a prob- 
lem, as these imply constraints on registration and rectification accuracies. 
Users also described the experimental design (e.g., -'mple segments or full- 
image area coverage) used to analyze a problem. 
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